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(54) Method and apparatus for replacing stuffing bits with supplementary data in an MPEG video 
data stream 

(57) An apparatus and method, applicable to varia- 
ble bit rate video and constant bit rate video, is dis- 
closed for replacing "stuffing bytes" with private data. 
The invention takes advantage of the otherwise wasted 
resources dedicated to -stuffing" in a data stream in 
order to insert private data. This is accomplished by 
inserting useful private data in a Transport Stream 
instead of the stuffing bits. That is, effectively, a re-mul- 
tiplexing operation occurs where, based on the exist- 
ence of certain conditions in a Transport Packet (e.g., 
stuffing bytes exist), the information necessary to 
replace stuffing bytes with private data yet still comply 
with established standards is accomplished. This data 
generally is referred to as privatestuff data in order to 
distinguish it from typical private data which may other- 
wise be encoded into a Transport Stream. The stuffing 
tytes removed from the Transport Packet may come 
from an adaptation field in the Transport Header or 
directly from the Transport Payload or both. 
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Description 

FIELD OF THE INVENTION 

s The present invention relates generally to data storage and transmission using MPEG standards and, more partic- 
ularly, the present invention relates to the established standards of transmitting "private data" and "stuffing bytes" in a 
Transport Data Stream complying with MPEG standards. 

BACKGROUND OF THE INVENTION 

10 

High Definition Television (HDTV) continues to make progress in its attempts to replace conventional television. 
Paving the way for this progress are various companies and associations working on standards to provide for a global 
market tor HDTV. 

One such group of companies is known as the "Digital HDTV Grand Alliance" including members such as AT&T 

is David Sarrtoff Research Center, Massachusetts Institute of Technology and others. A comprehensive overview of the 
strides made by this group are presented in an article by Robert Hopkins entitled "Digital Terrestrial HDTV for North 
America: The Grand Alliance HDTV System" published in the IEEE Transactions on Consumer Electronics (Summer 
1994). This article is herein incorporated by reference for all of its teachings regarding the background and basics of 
HDTV systems including the use of Program and Transport Packet Streams. 

so In addition to the Grand Alliance, much effort has been expended by the Moving Pictures Expert Group (MPEG), a 
committee within the International Standards Organization (ISO), in attempts to establish various standards for the stor- 
age and transmission of HDTV data (e.g., MPEG-2 standards - formats for Transport Packet Streams). Accepted stand- 
ards are periodically published such as the Video Section of Information Technology - Generic Coding of Moving 
Pictures and Associated Audio ISO/IEC 13818-2 (November 1994) (hereinafter Video Section'*) and the Systems Sec- 

25 tion of Information Technology • Generic Coding of Moving Pictures and Associated Audio ISO/IEC 1381 8-1 (November 
1994) (hereinafter "Systems Section") both of which are herein incorporated by reference for their teachings regarding 
established standards and formats including "stuffing" techniques. 

The syntax for the MPEG-2 standard defines several layers of data records which are used to convey both audio 
and video data For the sake of simplicity, the decoding of the audio data is not described herein. Encoded data which 

so describes a particular video sequence is represented in several nested layers, the Sequence layer, the Group of Pic- 
tures layer, the Picture layer, the Slice layer and the MacroWock layer. To aid in transmitting this information, a digital 
data stream representing multiple video sequences is divided into several smaller units and each of these units is 
encapsulated into a respective packefeed elementary stream (PES) packet For transmission, each PES packet is 
divided, in turn, among a plurality of fixed-length Transport Packets. Each Transport Packet contains data relating to 

35 only one PES packet The Transport Packet also includes a header which holds control information, sometimes includ- 
ing an adaptation field, to be used in decoding the transport packet. 

When an MPEG-2 encoded image is received, a transport decoder decodes the Transport Packets to reassemble 
the PES packets. The PES packets, in turn, are decoded to reassemble the MPEG-2 bit-stream which represents the 
image in the layered records, as described above. A given transport data stream may simultaneously convey multiple 

40 image sequences, for example as interleaved transport packets. This flexibility also allows the transmitter to switch 
among formats providing material in 4 by 3 aspect ratio according to one standard and widescreen (16 by 9) material 
according to another standard. 

Turning to a system implementation for delivering HDTV using MPEG-2 standards to the consumer, in general, as 
illustrated in high-level block diagram of Figure 1 , on the transmission side, video and audio signals are input to respec- 

4$ tive encoders 110 and 1 12, buffered in buffers 1 14 and 1 16, delivered to the system coder/multiplexer 1 18, and stored 
in storage unit 120 or transmitted by transmitter unit 120. On the receiving side, the signals are received by a system 
decoder/demultiplexer 122, again buffered in buffers 124 and 126, then decoded by decoders 128 and 130 and output 
as the original video and audio signals. 

An important aspect of the illustration of Figure 1 is that, although the intermediate stage buffering of the signals 

so includes a variable delay, the overall delay from input to output of the signals is required to be substantially constant. 
This is accomplished by monitored flow control and buffers. 

As indicated in Figure 1 . the delay from the input to the encoder to the output or presentation from the decoder is 
constant in this model, while the delay through each of the encoder and decoder buffers is variable. Not only is the delay 
through each of these buffers variable within the path of one elementary stream, the individual buffer delays in the video 

55 and audio paths differ as well. Therefore, the relative location of coded bits representing audio or video in the combined 
stream does not indicate synchronization information. The relative location of coded audio and video is constrained only 
by a System Target Decoder (STD) model such that the decoder buffers must behave properly; therefore, coded audio 
and video that represent sound and pictures that are to be presented simultaneously may be separated in time within 
the coded bit system by as much as one second, which is the maximum decoder buffer delay that is allowed in the STD 
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model. Similar to the STD model is a Video Buffering Verifier (VBV) which, as stated in the Video Section: 

Constant rate coded video bitstreams shall meet constraints imposed through a Video Buffering Verifier (VBV) 
defined in this clause.... 

The VBV is a hypothetical decoder! which is conceptually connected to the output of an encoder.. .Coded data is 
removed from the buffer as defined below. It is required that a Wtstream that conforms to this specification shall not 
cause the VBV to overflow. When low_delay equals 0, the bitstream shall not cause the VBV buffer to underflow... 

A high-level illustration of an exemplary STD model operating in conjuntton with an encoder is shown in Figure 2. 

The requirement that the VBV buffer or STD model decoders not underflow is very important as product quality is 
at stake. In order to maintain constant brtrate video, "stuffing" is implemented within various aspects of the system 
"Stuffing" is the act of filling the data stream with "donl care" Information sinply to maintain the required bit-rate 

For Transport Stream packets carrying PES packets, stuffing is used when there is insufficient PES packet data to 
fill the Transport Stream packet payload bytes to a level that would support the transmitted data rata 

Stuffing, for example, can be accomplished by defining an adaptation field longer than the sum of the lengths of the 
data elements in it, so that the payload bytes remaining after the adaptation field exactly accommodates the available 
PES packet data. The extra space in the adaptation field and/or payload can be filled with stuffing bytes. 

Figure 3 shows the format and field locations for a Transport Packet Stream where each Transport Packet includes 
a Header and a Payload The header of a Transport Packet includes fields for indicating the existence and controlling 
the length and content of an adaptation field. Within that adaptation field, another field is designated as "stuffing bytes". 
Stuffing bytes are similarly used in the payload of the Transport Packets. 

As mentioned, however, using stuffing bytes, which typically comprise all logical one values (i.e., "1 1 1 1 1 1 1 1") in the 
Transport Header and all logical zero values (i.a, "00000000") in the Transport Payload. is a waste of system resources 
(eg., transmission bandwidth). Accordingly, it would be desirable to make more efficient use of the system resources 
which, to date, have been limited to "stuffing". 



SUMMARY OF THE INVENTION 



The present invention, in a system including variable bit-rate video data in the form of data packets which uses 
stuffing bytes to fin a data stream, is directed to a method and system for removing the stuffing bytes and using the addi- 
tional bandwidth to transmit private data (hereinafter >ivatestuff data"). The invention includes examining means for 
examining a data packet which includes an indication of whether stuffing bytes are being used in the date packet and 
determining if the data packet is eligible, according to predetermined criteria, to have the stuffing bytes removed; and 
re-multiplexing means, responsive to the examining means, for removing the stuffing bytes from the data packet and 
adding predetermined privatestuff data to the data packet. 

In one aspect of the invention, the stuffing bytes are removed from a header portion of the data packet in order to 
gain additional transmission bandwidth. 

In another aspect of the present invention, the stuff ing bytes are rancved from a payload portion of the data packet 
in order to gain additional transmission bandwidth. In both aspects, however, the privatestuff data is inserted into the 
header portion of the data packet. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The invention is best understood from the following detailed description when read in connection with the accom- 
panying drawing, in which: 

Figure 1 (prior art) shows a Ngh-level block diagram of an exemplary digital multi-program transmission and receiv- 
ing system. 

Figure 2 (prior art) shows a high-level block diagram of an exemplary implementation of a STD model with portions 
of the system shown in Figure 1 . 

Figure 3 (prior art) shows an exemplary format, including field designations, for a Transport Packet Stream used in 
conjunction with the system shown in Figures 1 and 2. 

Figure 4A shows a high-level flowchart diagram illustrating the exemplary steps executed by the present invention 
to generally replace stuffing bytes with privatestuff data. 
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Frgure 4B shows a high-level flowchart diagram illustrating the exemplary steps executed by the present invention 
to replace stuffing bytes in the adaptation field with privatestuff data. 

Figure 5 shows a flowchart diagram illustrating the exemplary steps executed by the present invention to replace 
stuffing bytes in the packet payload with privatestuff data 

Figure 6 shows a flowchart diagram illustrating an exerrplary start code processing technique suitable for use with 
the embodiment shown in Figure 5. 

Figure 7 shows a flowchart diagram illustrating an exemplary stuffing bytes searching technique suitable for use 
with the embodiment shown in Figure 5. 

Figures 8A through 8C illustrate three examples of stuffing byte replacements as carried out by the techniques illus- 
trated in Figures 5-7. 

Figure 9 shows a high-level functional block diagram of an exemplary embodiment of an encoder suitable for use 
with the present invention. 

DETAILED DESCRIPTION 

As mentioned in the BACKGROUND section, for Transport Stream packets carrying PES packets, stuffing is used 
when there is insufficient PES packet data to fill the Transport Stream packet payload bytes to support the established 
data rate. One way stuffing can be accomplished is by defining an adaptation field that is longer than the sum of the 
lengths of the data elements in ft, so that the payload bytes remaining after the adaptation field exactly accommodates 
the available PES packet data. The extra space in the adaptation field is filled with stuffing bytes. Another way stuffing 
can be accomplished is by filling unused portions of the Transport Payload with zeros. 

The present invention, generally applicable to variable bit-rate video, takes advantage of the otherwise wasted 
resources dedicated to "stuffing" in order to insert private data. In the exemplary embodiment of the present invention, 
to take advantage of these otherwise wasted resources, useful private data is inserted in the Transport Stream instead 
of the stuffing bits. That is to say. effectively, a re-multiplexing operation occurs where, based on the existence of certain 
predetermined conditions in the fields of the Transport Stream (e.g„ stuffing bytes exist), the information necessary to 
replace stuffing bytes with private data yet still comply with the standard is generated and appropriately inserted. 

It should be noted that, although the present invention is descrfoed as being generally applicable to variable bit rate 
video, ft essentially is also applicable to constant bit rate video. That is to say, that, although. In the present invention, 
the modified video will always be variable bit rate video, the original video to be processed and transmitted may be 
either constant or variable bit rate video. 

The data that is used to replace the stuffing bytes generally win be referred to as >*ratestufP data in order to dis- 
tinguish it from typical private data which may otherwise be encoded into a Transport Stream. 

When privatestuff data is inserted in the Transport Stream, if necessary, rt can be sent with an individual program 
identification (PID) code indicating that the present Transport Packet includes privatestuff data. As described in the Sys- 
tems Section, a PID is a 13-bit field in a Transport Stream Header, indicating the type of data stored In the packet pay- 
load. Some PID values are assigned and some are reserved. In the exemplary embodiment of the present Invention, 
newly assigned PID values can be designated to indicate that the private data included in the particular Transport 
Packet is actually privatestuff data rather than normal private data. If a newly assigned PID is used, decoding of privat- 
estuff data may be easier on the receiving end. 

It should also be noted that, in addition to stuffing bytes, some Transport Packets are designated NULL packets 
using a special NULL PID. Using the techniques descrfoed herein, the present invention could also take advantage of 
the wasted resources of a NULL packet by remultiplexing the packet to include privatestuff data and all other appropri- 
ate fields (e.g., adaptation and private data fields). 

Additionally, as the stuffing bytes are onfy used on an "as needed" basis, the privatestuff data is sent on a "bursty" 
basis, i.e., only when the video channel "wants- to send stuffing bytes. Exanples of information which can be sent as 
privatestuff data include program reviews, program synopsis, etc for programs to be transmitted at a later time. 

As additional background, in the Systems Section, a syntax representation is provided for encocfing/decoding the 
adaptation field of a Transport Header. This syntax is represented below in Table I. 
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Table X *- Transport Stream adaptation field 



10 



IS 



so 



25 



Syntax 

adaptation^ ield () { 

adaptation_field length 

if (adaptation^ ield_length>0) { 
discontinuity_indicator 
random_access — indicator 
elementary_streamjpriority_indicator 
PCR_f lag ~ ~ * 

OPCR_flag 

splicing_point_f lag 
transportjprivate_data_flag 
adaptation^ ield_extension_f lag 
if (PCR_f lag » 'l' ) { 

program_clock_reference_base 

reserved 

program clock reference extension 

} 

if (OPCR_flag '1') { 

or i g inal_progr axn_cl ock_r e f e re n c e_ bas e 



Mo. of 
Bits 



Mnemonic 



8 


imsbf 


J 


bslbf 




bslbf 




bslbf 




bslbf 




bslbf 




bslbf 




bslbf 




bslbf 


33 


uimsbf 


6 


bslbf 


$ 


uimsbf 


33 


uimsbf 



30 



35 



40 



45 



SO 



6$ 



e 
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Table I (continued) 



Syntax 



10 



is 



so 



25 



30 



40 



reserved 

or iginal_p rogram_clock_ref e rence_ext ens i on 



No. of 
Bits 

6 
9 



} 

if <splicing_point_flag »= '1') { 
splice countdown 

} 

if (transport _private_data_f lag =«'!') { 
transport^private^data^length 
for (i«0; i< transporter i va te_data_l ength / i ++) { 
private data_byte 

} 

} 

if (adaptation_f ield_extension_f lag »» 'l'){ 
adap t at ion_f i e 1 d_ extens ion_length 
ltv_flag 

pie ce wi se_ra t e_f 1 ag 
seamless_splice_f lag 
reserved 

if <ltw_flag «'!'){ 

ltw_valid_flag 
lev offset 

} 

if (piecewise_ rate_ f lag '!'){ 
reserved 
piecewise rate 

if {seamless_splice_f lag »= 'l'){ 
splice_type 
DTS_next_au 122 . . 30] 
marlcer_bit 
DTS_next_au [29 . - 15] 
marker_bit 
DTS_next_au ( 14 . . 0] 
marker bit 

> 

for (i»0;i<N;i++) { 
reserved 

} 

) 

for (i-0) ;i<N;i+t-) { 
scuffing byte 

} 



8 
1 
1 
1 
5 

1 
15 



2 
22 



4 

3 

1 
IS 

1 
15 

1 



Mnemonic 

bslbf 
uimsbf 



tcimsbf 

uimsbf 
bslbf 



uimsbf 

bslbf 

bslbf 

bslbf 

bslbf 

bslbf 
uimsbf 



bslbf 
uimsbf 



bslbf 
bslbf 
bslbf 
bslbf 
bslbf 
bslbf 
bslbf 



bslbf 



bslbf 



so 



As shown in Table I, stuffing bytes are placed into the Transport Header in the adaptation field as needed. 

Referring to the syntax, the adaptation JieWJength, listed in Table I and illustrated in Figure 3. is an 8 bit field spec- 
ss ifying the number of bytes in the adaptation JiekJ immediately following the adaptation jieWJength. For example, in the 
exemplary embodiment, the value 0 is used for inserting a single stuffing byte in a Transport Stream packet. 

Moreover, when the adaptatk>njieldjx>ntrd value is '1 1\ the value of the adaptation JiekHength shall be in the 
range of 0 to 1 82. When the adaptattonJiekLcontrd value is '10', the value of the adaptation Jieidjength shall be 183. 

A stuff ingLbyte, for the adaptation field, is a fixed 8-M value usually equal to '1 1 1 1 1 1 11 • that can be inserted by 
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15 



20 



25 



theencoder. Once identified as stuffing bits, this "dent care" information is discarded on the reception end by the 
decoder. 

Continuing with Table I. in addition to the stuffing bytes, the syntax of the standard for the adaptation field provides 
for private data. For example, as shown in Figure 3. the two fields immediately before the stuffing bytes field are desia. 
n«ed "5 nags" and "optional fields". The "5 flags" field indicates if the optional fields existZ?s?me J££ ££ 
indicate the existence and length of transport private data". This same interrelationship of the fields is afeo presented 
in syntactical format of Table I. M'«»wirea 

In addition to the stuffing bytes used in the adaptation field of the Transport Header, as mentioned, stuffina bvtes 
may also be used in the Transport Payload. In the present invention, stuffing bytes from either the adaptation field or the 
^^P 0 ^ 08 ? 080 ** removed to prwWe addHfonal bandwid * r > ** privatestuff data, ft should be noted, however 
li olS^IL ^ 6 !!,' 6 femWed 'T eilher ,he adapta1ion ,ieW or the Trans P°rt Payload or both, the privatestuff 
data added to the packet, in the exemplary embodiment of the present invention, is onty added to an adaptation field in 
the Transport Header. «v««wiinoiuin 

Figure 4A shows a high-level flowchart illustrating exemplary steps executed for generally completing a stuffina 
byte removal and replacement operation, also known as a remultiplexing operation. This flowchart is intended to aen- 
erally illustrate stuffing byte replacement in either the adaptation field or Transport Payload 

«Jt^Z!£r 4Al ,ire V ^ <° 2 ' a Trans P°rt P«*et is captured from the Transport Stream and. then, at 
step 404. the Packet is examined. The examination includes determining if stuffing bytes exist, step 405 and if so 
where and how many, step 407. Then, using information obtained during the examination, at step 408. a remurfolaxinq 
operation occurs. That is, the stuffing bytes are replaced with privatestuff data Additional details for the above stens 
are provided below with reference to Figures 4B. 5-7 and 8A-8C. 

Figure 4B shorn s a high-level flowchart diagram illustrating the exemplary steps executed by the present invention 
to replace stuffing bytes m the adaptation field with privatestuff data. As shown in Figure 4B. first, at step 410 a Trans- 

ZSZZtlZZTE? ^ ? Tra H SP ° rt Stream andl ta ' at step 412 ' *• Packet «amined. The examination 
ISSflTS? d ? erm,r "?<| "I an a 1 dap,at ° n ,ield •** 8tep 4U - accomplished by examining various 

fields. Next, rf there is an adaptation field, then it is determined if private data exists in the adaptation field, step 416 In 
the exemplary embodiment this can be accomplished by examining the "5 flags" field and "transport private data 
length-field. If private data exists, the process ends because this adaptation field, in the exemplary embodiment of the 
present "nrnfton. will not be used for privatestuff data. It should be noted, however, that although it is possible to insert 
prrvBte^uff data when private data already exists, the present invention intentionally elects to not disturb an adaptation 
field which already contains private data. ^ 

Continuing with the flowchart of Figure 4B. if private data does not exist, the location and number of stuffing bytes 
■s^emwied, at step 418. using the information from the various fields and. at step 420. a remultiplexing operation 

P ro ^ edi "9 ,0 OT « ;«wWp'exing operation, it is important to remember that any modification of the adaptation 
2S? S e estat * shed standards (i.e.. the formats shown in Figure 3 and the syntax Gsted in Table O^There- 

fore. ifstuff ins i bytes in an adaptation field are to be replaced with privatestuff data, not only is the privatestuff data mul- 
tiplexed into the data stream but an of the appropriate bits in the appropriate fields are set accordingly. For example if 
in a particular adaptation field, no optional fields had existed in the initial examination, the *S flags" field is modified to 
T Z!^. if er me rfHm " ti P ,ex operation, the optional fields do exist Furthermore, the number of privatestuff bytes is 
added to the "transport private data length" field in order to properly indicate the modification of the adaptation field 

Being aware of the established standards such as those herein incorporated by reference, one of ordinary skifi in 
tlw art would appreciate the various combinations of fields which may exist when attempting to replace stuffing bytes 
with privatestuff data, thereby, understanding that all necessary fields would have to be modified during the re-multiplex- 
ing operation to reflect the updated content of the adaptation field. P 
„ Notwithstanding aMfc'es of one of ordinary skill in the art. by way of example. Figures 5-7 and 8A-8C show flow- 

^^P^^^f detei,in9 ** 8tepS nece$sary to detect 'amove stuffing bytes from a single 

Transport Payload and utilize the consequent extra bandwidth to transmit privatestuff data in an adaption field at the 
transport layer. 

• It should be noted that the method, illustrated in Figures 5-7 and 8A-8C, 

^ <i "* the ™" im " m r f» uired number of video stuffing bytes within a transport packet based on the structure of its 
adaptation field (described above), 

2)tocates the positions of video stuffing bytes within a transport payload and removes these bytes from the pay- 
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3) inserts private data at tfie adaptation field, and 
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4) locates a picture header structure and replaces its VBVDELAY value with OxFFFF. 



It should be further noted that the method illustrated in Figures 5*7 and 8A-8C assumes that 

1) the program association table (PAT) and program map table (PMT) have been processed and the PID tor the tar- 
geted video elementary stream has been recognized and 

2) the parsing of the payload of the Transport Payload containing the video elementary stream starts at or before 
the very first video sequence header and ends at the sequence end code. 

Turning to Figure 5-7, the illustrated processes examines many fields within the Transport Header and Payload as 
well as the tracking of some field sires. For that purpose, variables are used throughout the process. In particular, the 
variable legends provided below tor each of Figures 5, 6 and 7, show the particular variables used for that flowcharter*! 
their corresponding definitions: 



Variables Used in Figure 5 



CnLV_ZB » Counter of video zero bytes 

StrtCd_Fnd « Flag to indicate a finding of a start code 

PictStrtCd.Fnd - Flag to indicate a finding of a picture start code 

MIN_V_SB = Minimum number of video stuffing bytes that need to exist in the TP packet 

CntV.SB « Counter of video stuffing bytes 

Cnt_Pyld_B * Counter of the TP payload bytes 

Variables Us ed in Fioure 6 



PictStrtCd.Fnd = Flag to indicate a finding of a picture start code 
StrtCd.Fnd = Flag to indicate a finding of a start code 
Cnt_PicLHdr = Counter of bytes in a picture header structure 



Variables Used in Fioure 7 



Cnt_PyW_B -Counter of the TP payload bytes 

Cnt_V_SB = Counter of video stuffing bytes 

CnLY-ZB b Counter of video zero bytes 

Cnt_Cur_SB = Counter of current video stuffing bytes 

StrtCd_Fnd = Rag to indicate a finding of a start code 



Referring to Figure 5, at steps 510-514, the variables "CnLV.ZB". "StrtCd_FncT and "PctStrtCd.Fnd", which are 
used during processing, are initialized. At steps 516 and 518, preprocessing is performed to detect the next sync byte 
and verify that the PID is for an elementary video program. 

At steps 520, 522 and 524, the process examines the adaptation field control and length fields and, depending on 
the results of the examination, sets a variable indicative of the minimum number of stuffing bytes which need to exist in 
the Transport Packet, steps 526 and 528. 

Next, at step 530, it is determined whether there is any private data in this adaptation field. And, again, based on 
this determination, the process sets a variable indicative of the minimum number of stuffing bytes which need to exist 
in the TP packet, step 532. Here, as mentioned, if it is determined that private data already exists in the adaptation field, 
then the process terminates and starts anew. 

The process of Figure 5, at step 536, then jumps to the first byte of the payload while, at step 538, initializing the 
variables w Cnt_V_SB" and "CnLPyMB". 

Now. step 540 including its finks to thef lowcharts of Figures 6 and 7, essentially corresponds to step 404 in Figure 
4A. It is here that the number and location of stuffing bytes is determined. The flowchart of Figure 6 detects and tracks 
start codes in the payload while the flowchart of Figure 7 detects and counts stuffing bytes. Tracking where and how 
many bytes of each exist, allows the process to remove the stuffing bytes (thereby providing additional bandwidth for 
privatestuH data) and preserve the picture data. At step 541, GnLVJBB is checked to ensure that it is greater than 
MIN_V_SB. The purpose of step 541 is to ensure that the number of found stuffing bytes in this Transport Packet is 
larger than the minimum number of stuffing bytes that are required to exist in order to follow through with the remuW- 
plaxing. 

finally, at steps 542 and 544 (which generally correspond to step 408 of Figure 4A), the stuffing bytes are removed 
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and the re-multiplexing of privatestuff data atong with appropriate control field modifications, occurs, respectively. This 
re-multiplexing operation is further illustrated, as indicated at step 544) by way of example in Figures 8A. 8B and 8C 

Referring to Figure 6, the process, knowing the format of start codes used by meaningful data (e g picture data) 
in the payload of a Transport Packet, counts the stuffing bits in the packet payload until a start code is encountered A 
5 syrtoforthevartousstartcaJesispresertedintheVKJeo 
data bang earned by the Transport Packet Payload is disturbed. 

As seen in steps 610 and 612, the process in Figure 6. as long as either the PictStrtCd Fnd or the StrtCd Fnd is 

K?h£ L^^S te ^!! ]Z l^r^ lhe TranSp ° rt Payt0ad 009 byte at a time urtjTthe condition exisTwhere 
both the PKtStrtCd Fnd and the StrtCd.Fnd are equal to v. When this condition is detected, the process shifts to the 

10 steps illustrated in Figure 7 which attempts to track and identify the stuff ing bytes. 

Until then, if PictStrtCd.Fnd is equal to T then the subsequent picture header bytes are tracked and processed 
steps 614. 616. 618. 620. 622. 624. 626. and 628. until PtetStrtCd.Fnd is finally reset at step 630 And if 
P«ctStrtCd_Fnd is equal to 0' but StrtCd_Fnd is equal to '1 1 then, trie current byte of data is processed, steps 632 634 
636 and 638. such that either the PictStrtCd.Fnd is set, step 634. or the StrtCd.Fnd is reset step 638 

'5 it should be noted that the states of the variables PictStrtCd.Fnd and StrtCd.Fnd can reflect a condition that was 
processed and determined in a previous Transport Payload. That is to say. that picture data or stuffing bytes can overlap 
more than one Transport Payload. therefore, if the previous Transport Payload ended with a start code, the processing 
for the present" Transport Payload should take that into consideration as Transport Packets are simply portions of a 

« vS?"/* e T pte ' cona^g *ree consecutive Transport Packets, stuffing may begin rnidway through 

eo the Payload of the first packet, continue for the entire Payload of the second packet and end midway through the third 
packet This type of condition should be considered during stuffing byte detection and removal. 

Referring to Figure 7. the process searches for stuffing bytes, carefully checking the value of all bytes and tracking 
the location of vanous points used to identify and. subsequently, remove the stuffing bytes. Briefly, steps 710 and 712 
process and count the zero bytes that are encountered. Eventually the byte to be processed will not be OxOO' cor- 

* H 2 * prth fr0m *" 71 °' and ,ikely be ™ )1 * lndicatin 9 a start code has been found, corresponding 

to the YES exrt path of step 716. At this point the StrtCd.Fnd variable, which is used in Figure 6. is set step 718 and 
the stuffing byte counting stops. The actual number of stuffing bytes is determined by subtracting 2 from the CntV ZB 
step 720. because a start code includes twenty three logical zeros and a logical one. However, step 716 only checks 

„„ j^' 09 "* 1 zeros and a l0 9 ical °° e - therefore, sixteen more logical zeros (or two bytes of 'OxOOOO') are not consfcf- 
30 erea stuffing bytes. 

Consequently, if the number of stuffing bytes. Cnt_Cur_SB is less than the payload byte count step 722. and not 
rJ 0 ^^ 4, ttle loca,ion and amount °* stuffing bytes is calculated and recorded using the variables such 
as Cnt_Pyfd B and Cnt.V.ZB which had been used to count and mark important points in the payload processing ft 

as tS^JSl U * e " UWb9r . * **** calculated durin 9 •* processing is then added, at step 728. to 

» the number of stuffing bytes previously recorded. 

Additional discussion of figures 6 and 7 rs not warranted as one of ordinary skill in lhe art. having the description 

BUfeS 5 'L and8A - 8Cas weH ^ the knowledge of established standards before hto finduding Ze 
incorporated herein, can understand processes illustrated in Figures 6-7. 

<n h~2Z?* t^* 80 S C°r. examp,es of Trans P° rt Packets before and after being processed by the present invention 
« described n Figures 5-7. tn particular. Figure 8A shows field replacement diagrams (he,, original and processed) for a 
Transport Packet which originally had no adaptation field. This corresponds to lhe YES path exiting from step 520 in 
Figure 5. As shown in Figure 8A, the Transport Packet Payload is stripped of its stuffing bytes, an adaptation field with 

1 J!2^J! 1 ?5? "rf 5 !^ to indicate 3 Transpor1 Header P™* 8 is created and the packet is recon- 
structed such that the Payload no longer includes stuffing bytes. As seen in the legend below the depiction of the proc- 
f^^ Tfa ^rt Packet, many of the values placed in the newly formed fields (e.g.. adaptation field length) are derived 
from the state of vanables used during the processing in Figures 5-7 (e.g.. Cnt_V_SB). 

Figure 8B illustrates the case where the Transport Packet included an adaptation field but the length of the adapta- 
toon f .eld was 0 bytes. This corresponds to the YES path exiting from steps 522 and 524 of Figure 5. Again the Trans- 

«. ^T^J 5 ^T? 01 1he ., Stuf,in9 bytes ^ the appropriate adaptation fields are created and/or modified 
so according to the value of a variable. 

Figure 8C illustrates yet another example where an adaptation field existed and the adaptation field length is not 
equal to zero and there is no private data. This corresponds to the NO path exiting from step 530 of Figure 5. Again 
similar stuffing byte removal and adaptation field modifications occur. 

« nra^SZl^f V*** d ™ exenplary <* a pnVatestuff processor suitable for use with the 

a T^^L flure 9 ' *" Transport stream fe caP*"* 1 >" buffer 910 as well as monitored by an analyzer 912 
Analyzer 912 performs much of the processing corresponding to steps 402-404 of Figure 4A and steps 41 0-41 8 of Fig- 
ure 4B indudmg.for payload processing, the steps in Figures 5-7 except step 544. Next, analyzer 912 instructs remul- 
tiplexer 914 whether or not to perform a remultiplexing operation and with what information. Remulfjplexer 914 performs 
the processing corresponding to steps 408. 420 and 544. Again, the Transport Stream provided by buffer 910 is tempo- 
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rarity delayed by buffer 916. Buffers 910 and 916 are generally used to compensate for the processing delays of ana- 
lyzer 912 and remultiplexer 914, respectively. Controller 918 performs miscellaneous control operation including 
controling the data flow through the buffers and ultimately deciding whether the Transport Packet will pass through mul- 
tiplexer 920 or whether the processed output of remultiplexer 916 will pass. 

It should be noted that, although only the general flowchart for processing the stuffing bytes in an adaptation field 
is provided (Figure 4B), one of ordinary skill in the art, using the information provided and known, could easily detect 
and removing the stuffing bytes from the adaptation field and creating or modifying fields in the adaptation field. For 
instance, a field replacement diagram for this case would appear quite similar to Figure 8C except the original Transport 
Packet would include stuffing bytes before the TP Payload and the processed Transport Packet would include new pri- 
vate data (i.e. privatestuff data in the appropriate private data location) and no stuffing bytes in the adaptation field 
Moreover, the lield locations described in Table I and illustrated in Figure 3 provide the details necessary to detect and 
remove stuffing bytes from the adaptation lield. 

Although the invention is illustrated and described herein as embodied in a method and apparatus for replacing 
stuff »ig bytes with privatestuff data in an MPEG encoded data stream, the invention is not intended to be limited to the 
details shown. Rather, various modifications may be made in the details within the scope and range of equivalents of 
the claims and without departing from the spirit of the invention. 

Claims 

1 . In a system including constant bit-rate video data and variable bit-rate video data in the form of data packets which 
uses stuffing bytes to fill a data stream, a 6ystem for replacing the stuffing bytes with privatestuff data comprising: 

means for analyzing a data packet which includes an indication of whether stuffing bytes are being used in the 
data packet and determining if the data packet is eligble, according to predetermined criteria, to have the stuff- 
ing bytes removed; and 

re-multiplexing means, responsive to the examining means, for removing the stuffing bytes from the data 
packet and adding predetermined privatestuff data to the data packet 

2. The system according to claim 1, wherein the data packet includes a header portion and a payload portion and 
stuffing bytes are located in the header portion. 

3. The system according to claim 1. wherein the data packet includes a header portion and a payload portion and 
stuffing bytes are located in the payload portion. 

4. The system according to claim 1 , wherein the data packet includes a header portion and a payload portion, wherein 
the data packet further includes an indication as to whether private data is being carried by the header portion of 
the data packet wherein the predetermined criteria is that no private data is being carried by the header portion of 
the data packet 

5. The system according to claim 1 , wherein the data packet includes a header portion and a paytoad portion and the 
privatestuff data is inserted In an adaptation field in the header portion. 

6. In a system including constant bit-rate video data and variable bit-rate video data in the form of data packets which 
uses stuffing bytes to fill a data stream, a method for removing the stuffing bytes from a data packet to create addi- 
tional bandwidth and using the additional bandwidth to transmit privatestuff data conprising: 

analyzing a data packet which includes an indication of whether stuffing bytes are being used in the data 
packet and determining if the data packet is eligible, according to predetermined criteria, to have the stuffing 
bytes removed; 

responsive to the analyzing step, removing the stuffing bytes from the data packet to create additional trans- 
mission bandwidth; and 

adding predetermined privatestuff data to the data packet thereby using the additional transmission bandwidth. 

7. The method according to claim 6, wherein the data packet includes a header portion and a payload portion, and 
the stuff ing bytes are removed from the header portion. 
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8. The method according to daim 6. wherein the data packet includes a header portion and a payload portion stuffing 
bytes are removed from the paytoad portion. ' * 

9. The method according to claim 6, wherein the data packet includes a header portion and a payload portion 
wherein the data packet further includes an indication as to whether private data is being carried by the header por- 
tion of the data packet, wherein the predetermined criteria is that no private data is being carried by the header por- 
tion of the data packet. 

10. The method according to claim 6, wherein the data packet includes a header portion including an adaptation field 
and a payload portion, the privatestufl data is added by insertion into the adaptation field. 

1 1. In a system including constant bit-rate video data and variable bit-rate video data in the-form of data packets which 
uses stuffing bytes to fill a data stream, a system for replacing the stuffing bytes with privatestufl data comprising: 

means for analyzing a data packet which includes an indication of whether stuffing bytes are being used in the 
data packet; and 

re-multiplexing means, responsive to the examining means, for removing the stuffing bytes from the data 
packet and adding predetermined privatestufl data to the data packet 

1 2. The system according to claim 1 1 . wherein the data packet includes a header portion and a payload portion and 
stuffing bytes are located in the header portion. 

13. The system according to daim 11. wherein the data packet includes a header portion and a payload portion and 
stuffing bytes are located in the payload portion. 

14. The system according to claim 1. wherein the data packet includes a header portion and a payload portion and the 
privatestuff data is inserted in an adaptation field in the header portion. 

1 5. In a system including constant bit-rate video data and variable bit-rate video data in the form of data packets which 
uses stuffing bytes to fin a data stream, a method for removing the stuffing bytes from a data packet to create addi- 
tional bandwidth and using the additional bandwidth to transmit privatestuff data comprising: 

analyzing a data packet which includes an indication of whether stuffing bytes are being used in the data 
packet; 

responsive to the analyzing step, removing the stuffing bytes from the data packet to create additional trans- 
mission bandwidth; and 

adding predetermined privatestuff data to the data packet thereby using the additional transmission bandwidth. 

16. The method according to claim 15. wherein the data packet includes a header portion and a payload portion, and 
the stuffing bytes are removed from the header portion. 

17. The method according to daim 15. wherein the data packet includes a header portion and a payload portion, stuff- 
ing bytes are removed from the payload portion. 

18. The method according to daim 15. wherein the data packet indudes a header portion including an adaptation f ield 
and a payload portion, the privatestuff data is added by insertion into the adaptation field. 
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